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A CM I MODEL FOR AN INDIVIDUALIZED LEARNING 
PROGRAM IN NINTH GRADE SCIENCE 



Robert P. O'Reilly 
New York State Education Department 

and 

Ronald K. Hambleton 

University of Massachusetts 

The project on which this report is based is supported by the New 
York State Education Department, under the aegis of Experimental Progx-ams 
directed by Robert P. O’Reilly. The project is directed by Olcctt Gardner, 
Director of Research, Jamesville-DeWitt Central Schools, DeWitt, New York. 
Principal consultants for the project are Robert P. O’Reilly and Howard 
Berkun, Division of Research, New York State Education Department, end 
Ronald K. Hambleton of the,, School of Education, University of Massachusetts 

The Jamesville-DeWitt (JD) project is part of a series of efforts 
designed to introduce into New York State Schools new generalizable systems 
for refining instructional programs and for managing and reporting individ- 
ual student progress. The JD system is in part an effort to test the feasi 
bility of managing the instructional process and the learning process of 
individuals via a small, third-generation computer installed in the school. 

Rationale 

Our first attempts to initiate new approaches to evaluation in the 
schools focused on implementing the Comprehensive Achievement Monitoring 
(CAM) model. Implementation of this model is based on the following com- 
ponents: 

1 . Defining course objectives in behavioral terms ; 

2 . Preparing several test items to measure each instructional 
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objective, usually three to five ; 

3. Organizing test items into a series of parallel forms, 

in which the objectives are equally represented in each test 
form; 

4 . Administering all test forms on each test occasion . 

(Since CAM testing takes place periodically — usally at 
two or three week intervals— —by design it is usually 
possible to have each student take each test form once 
during the course)^ 

5 . Analyzing and reporting of test results to students, 
teachers, and administrator s, (The test scoring and 
analysis functions of CAM are computerized.) 

The system provides an appropriate data base for decision types 2, 

3, and 5 listed in Table 1 and 2. Essentially CAM is a formative evaluation 
technique designed to support the refinement of instructional programs, 
but is also useful for measuring student progress in a course (Gorth, 
Schriber and O’Reilly, 1970; Hambleton, 1971; Hambleton, Gorth and 
O’Reilly, 1971). 

Experience indicates that teachers can usually be interested in 

h 

implementing CAM as a course refinement technique, and with appropriate 
assistance, will perform the relatively arduous supporting tasks of pre- 
paring objectives and items, structuring their curricula, and other 
necessary activities. There are, however, numerous additional problems 
in implementation which prevent full systematic use of the technique. 

Among them are: 

Vaguely defined instructional models or models used for instruction 
are only vaguely understood by teachers; 

Turn-around time is frequently inadequate due to problems with data 
logistics; 
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*>Iastery Test: Pre- and Post Testing, remedial Post- 

*CET: Single items embedded in curriculum materials which provide a partial signal on daily progress 
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3. Cost of data processing and analysis in external facilities: and 

4. Inadequate teacher training in the use of criterion-referenced 
evaluation systems; curriculum design, instructional design, etc. 

Programs 2 and 3 above become even more constraining when the infor- 
mation needs of an individualized program of assessment are considered. 

For program refinement, testing is primarily on the terminal objective 
level and delays of one to two weeks in processing can be tolerated. 

Individual assessment, however, implies more or less continuous tracking 
of performance at the course objective level, and a system which will deliver 
data for decision-making on a day-to-day basis. This also represents a 
substantial increase in information processing over-the requirements of 
CAM data. Typical batch-mode computer service in external facilities is 
ordinarily inappropriate for individual assessment and barely appropriate for 
the CAM analysis; the time-share mode has not been proved economically feasible 
for either form of analysis. 

The problems noted with the CAM system and projected individual 
assessment programs seem to be adequately met in the JD system, which is 
based on a generalizable curriculum model and uses a small, in-house, high- 
speed computer. Prior experience with this processing mode indicated that 
costs are substantially below time-share and external batch modes. Turn-around 
time is dramatically improved, as the schools involved in-effect begin 
operating their own data processing centers. There is, however, much 
developmental work remaining for the JD system, particularly the test of 
the individualized evaluation system to be described. 

The Instruction Model 

The 9th grade science course through which the JD model is being 
implemented is organized into "modules" which consist of a series of instruc- 
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tional events arranged in an assumed hierarchy of behavioral objectives leading 
to mastery of a single concept or group of related concepts. Each module 
employs a variety of instructional modes encompassing the spectrum from 
large group lectures to independent reading assignments. The day to day 
instructional activities which, when taken together make-up a module, are 
organized into a hierarchy of smaller units called learning activity 
packages (LAPs) . 

A typical module begins with a pretest which is used as a basic diagnostic 
tool along with measurements of a student’s social relationships, interests 
and overall acheivement to prescribe a sequence of LAP’s for the student. 

Each LAP is designed around the smallest number of behavioral objectives 
possible and secondary diagnosis is built into each LAP tc provide both self- 
testing by the student and criteria for a higher level of diagnosis by the 
teacher . 

The flow of the process Continuously returns to teacher diagnosis 
which places the teacher in a role of designer and manager of learning 
opportunities for students in independent learning situations, rather than 
as the provider and dispenser of what is to be learned. Within this frame- 
work the teacher has a significant amount of direct contact with each indi- 
vidual student, since he is responsible for diagnosing a student’s learning 
problems and then guiding him to learning activities and materials which will 
meet his needs. The teacher also maintains some of his traditional functions 
in that he aids in small group discussions and takes charge of instruction 
in large group settings. 



Program Refinement Using CAM 

Subjects for the current year of developmental work are 350 ninth 
graders of both sexes. All subjects are participating in a CAM testing 
program and the individual testing program, the latter of which is in its 
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early developmental stages . 

The CAM application focuses on 80 objectives, identified as the 
terminal behaviors in 13 instructional modules. For each objective, 
four alternative multiple choice items were constructed. Using stratified 
random sampling, the items were used to construct eight parallel test forms. 

The total pool of 320 items divided into the eight test forms is scheduled 
for administration at eight equally spaced intervals during the course of 
the year; each test form is assigned on a random basis to each student, with 
no duplication of administration of forms. 

Data for each CAM testing are now being punched on cards and processed 
via the CAM computer program and CDC 3600 at the University of Massachusetts . 
Results of the analysis are returned in about 10 days of the testing. The 
results for individuals include right-wrong scores identified by objective 
and total score. Individual results are updated at each analysis. The data 
for individuals are further summarized as group data in the form of raw scores 
and difficulty levels. Samples of computer output made available to students 
and teachers after each test occasion are provided by Hambleton, Gorth and 
O’Reilly (1971). The individual and group data from this CAM application are 
specifically appropriate for two of the decision classes identified in Table 
2: (1) Type 3 , program refinement; and (2) Type 5 , determining student progress 

on terminal course objectives. 

To improve turn-around time for the CAM analysis, and to provide the 
rapid turn-around time required for the individualized testing system, a 
Digital Equipment Corporation PDP-12B computer, with a BA-12 peripheral 
expander, card reader, teletype, keypunch and mark-sense input capabilities 
has been installed in the school. The PDP-12B has the capability of scoring 
200, 20-item tests per minute and recording all student responses and summary 

i 

data on magnetic tape fot‘ retrieval. This machine configuration is now being 
programmed for the individualized testing system. A mark-sense card procedure 
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has been designed for use in the machine and has received an initial test. 

In a similar experimental program, the CAM pupil and course profile analysis 
capability has been translated into programs which have been running in 
the PDP-12. These programs are now being modified for use in the JD system 
and will shortly replace analysis via the CDC 3600 facility. Primary output 
from the CAM analysis in the PDP-12 will consist of raw scores identified 
by objective for each student and raw scores and difficulty levels by 
objective across individuals. Profiles will be obtained by cutting and 
pasting the results of individual monitors. 

The Individualized Testing Model 

In progress is the development of a decision-making model which 
will provide the guidelines for individual assessment. Within each 
instructional module, there exist five major decision-making points. To 
provide information for decision-making at 4 of these points the following 
tests are administered: module pretest, module post test, lap pretest, 

and lap post test. At the fifth point, a combination of information including 
a student autosurvey, direct observation by teachers, and curriculum embedded 

tests are used. (It is also important to note that this information which 

\ 

is collected for student assessment is also used for formative evaluation.) 

Briefly let us consider each decision point separately.. , As each student 
begins to work on a module, a pretest is administered. Since items in the pre- 
test are closely tied to the general objectives of the LAPs, on the basis of 
the student’s successful performance on some of the items it is possible to 
omit the corresponding LAPs from the student’s prescription for the module. 

Such a procedure will insure that students will be working only on learning 
experiences directed toward goals which have not been mastered previously. 

The module post test which is either the same test or a parallel form of the 
module pretest can be used for prescribing remedial work for a student. 
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for grading, and for evaluating the effectiveness of instruction in 
the LAPs. 

As with the module pretests, LAP pretests are used to prescribe 
a set of objectives within the LAP that the student must demonstrate com- 
petency in before moving on to the next LAP in his prescription. LAP Dost 
tests are used to determine the extent to which students have satisfactorily 
completed the objectives of the LAP. 

As the student proceeds through a LAP he is tested continuously on 
the subject matter. When the student does not perform up to expectations, 
remedial work is prescribed. Since there is a wide variety of learning 
disabilities, a great deal of attention is being directed toward diagnostic 
methods and the development of remedial programs. 

At the present time, the computer is being used to record student progress 
on a daily basis as well as to score tests. Daily computer reports to the 
teachers will be extremely useful for diagnostic purposes, since they will 
enable the teacher to know the position of each individual in the LAP, achieve- 
ment progress by each student in the LAP, as well as individual achievement 
ih the module. Eventually the computer will also be used to prescribe programs 
of study for the students. 

At our present stage of development there are still many important questions 
to answer. For example, since the LAPs within a module, as well as the objectives 
within a LAP can be arranged in hierarchies, is it possible to develop tailored 
testing procedures for the module and LAP pretests? If applicable, tailored 
testing would undoubtedly prove to be more efficient than the present tasting 

arrangement. How much information about a student is needed to optimally 
prescribe a program for him? Could we incorporate testing procedures which 
would reduce the effect of guessing on test scores? 

Expected Benefits 

There are a number of cost-effective components in the Jamesville-DeWitt 
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IPI program. The first of these is found in the computer system which 
generates, on a timely basis, data on programs and individuals for adminis- 
trators, teachers and students at a cost considerably below comparable 
data provided by facilities external to the school. Secondly, the higher 
student to teacher ratio in the IPI science program produces additional 
savings relative to conventional classroom instruction. Instructional costs 
are further reduced by the introduction of the computer which performs many 
of the functions that are typically handled by teacher’s aides in other IPI 
programs. Finally, the programmed format of the instructional materials 
produces greater efficiency in terms of average time required for a student 
to complete course requirements. 
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